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Abstract 

Social and psychological interventions are often complex. Understanding randomized controlled trials (RCTs) of these complex 
interventions requires a detailed description of the interventions tested and the methods used to evaluate them; however, RCT 
reports often omit, or inadequately report, this information. Incomplete and inaccurate reporting hinders the optimal use of 
research, wastes resources, and fails to meet ethical obligations to research participants and consumers. In this article, we explain 
how reporting guidelines have improved the quality of reports in medicine and describe the ongoing development of a new 
reporting guideline for RCTs: Consolidated Standards of Reporting Trials-SPI (an extension for social and psychological interven- 
tions). We invite readers to participate in the project by visiting our website, in order to help us reach the best-informed 
consensus on these guidelines (http://tinyurl.com/CONSORT-study). 
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Introduction 

Social and psychological interventions aim to improve physical 
health, mental health, and associated social outcomes. They are 
often complex and typically involve multiple, interacting inter- 
vention components (e.g., several behavior change techniques) 
that may act and target outcomes on several levels (e.g., indi- 
vidual, family, and community; Medical Research Council 
[MRC], 2008). Moreover, these interventions may be contex- 
tually dependent upon the hard-to-control environments in 
which they are delivered (e.g., health care settings and correc- 
tional facilities; Bonell, 2002; Pawson, Greenhalgh, Harvey, & 
Walshe, 2004). The functions and processes of these interven- 
tions may be designed to accommodate particular individuals 
or contexts, taking on different forms while still aiming to 
achieve the same objective (Bonell, Fletcher, Morton, Lorenc, 
& Moore, 2012; Hawe, Shiell, & Riley, 2004). 

Complex interventions are common in public health, 
psychology, education, social work, criminology, and related 
disciplines. For example, multisystemic therapy (MST) is an 
intensive intervention for juvenile offenders. Based on social 
ecological and family system theories, MST providers target a 
variety of individual, family, school, peer, neighborhood, and 
community influences on psychosocial and behavioral problems 
(Henggeler, Schoenwald, Rowland, & Cunningham, 2002). 
Treatment teams of professional therapists and caseworkers 
work with individuals, their families, and their peer groups to 
provide tailored services (Littell, Campbell, Green, & Toews, 
2009). These services may be delivered in homes, social care, 



and community settings. Other examples of social and psycholo- 
gical interventions may be found in reviews by the Cochrane 
Collaboration (2013; e.g., the Developmental, Psychosocial, and 
Learning Problems Group; the Cochrane Public Health Group) 
and the Campbell Collaboration (2013). 

To understand their effects and to keep services up to date, 
academics, policy makers, journalists, clinicians, and consu- 
mers rely on research reports of intervention studies in scien- 
tific journals. Such reports should explain the methods, 
including the design, delivery, uptake, and context of interven- 
tions, as well as subsequent results. Accurate, complete, and 
transparent reporting is essential for readers to make best use 
of new evidence, to achieve returns on research investment, 
to meet ethical obligations to research participants and consu- 
mers of interventions, and to minimize waste in research. 
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However, reports of randomized controlled trials (RCTs) are 
often poorly reported within and across disciplines including 
criminology (Perry, Weisburd, & Hewitt, 2010), social work 
(Naleppa & Cagle, 2010), education (Torgerson, Torgerson, 
Birks, & Porthouse, 2005), psychology (Michie et al., 2011; 
Stinson, McGrath, & Yamada, 2003), and public health 
(Semaan et al, 2002). Biomedical researchers have developed 
guidelines to improve the reporting of RCTs of health-related 
interventions (Schulz, Altman, & Moher, for the CONSORT 
Group, 2010). However, many social and behavioral scientists 
have not fully adopted these guidelines, which may not be 
wholly adequate for social and psychological interventions in 
their current form (Bonell, Oakley, Hargreaves, Strange, & 
Rees, 2006; Davidson et al., 2003; Perry et al., 2010; Stinson 
et al., 2003). Because of the unique features of these interven- 
tions, updated reporting guidance is needed. 

This article describes the development of a reporting guide- 
line that aims to improve the quality of reports of RCTs of social 
and psychological interventions. We explain how reporting 
guidelines have improved the quality of reports in medicine, and 
why guidelines have not yet improved the quality of reports in 
other disciplines. We then introduce a plan to develop a new 
reporting guideline for RCTs — Consolidated Standards of 
Reporting Trials (CONSORT)-SPI (an extension for social and 
psychological interventions) — which will be written using 
recommend techniques for guideline development and dissemi- 
nation (Moher, Schulz, Simera, & Altman, 2010). Wide stake- 
holder involvement and consensus are needed to create a 
useful, acceptable, and evidence-based guideline, so we hope 
to recruit stakeholders from multiple disciplines and professions. 

Randomized trials are not the only rigorous method for 
evaluating interventions; many alternatives exist when RCTs 
are not possible or appropriate due to scientific, practical, and 
ethical concerns (Bonell et al., 2011). Nonetheless, RCTs are 
important to policy makers, practitioners, scientists, and ser- 
vice users, as they are generally considered the most valid and 
reliable research method for estimating the effectiveness of 
interventions (Chalmers, 2003). Moreover, many of the issues 
faced in reporting RCTs also relate to other evaluation designs. 
As a result, this project will focus on standards for RCTs, which 
could then also inform the development of future guidelines for 
other evaluation designs. 

Impact of CONSORT Guidelines 

Reporting guidelines list (in the form of a checklist) the mini- 
mum information required to understand the methods and 
results of studies. They do not prescribe research conduct, but 
facilitate the writing of transparent reports by authors and 
appraisal of reports by research consumers. For example, the 
CONSORT Statement 2010 is an evidence-based guideline; 
to identify items, the developers reviewed evidence of trial 
design and conduct that could contribute to bias. Using consen- 
sus methods, they developed a checklist of 25 items and a flow 
diagram (Schulz et al., 2010). CONSORT has improved the 
reporting of thousands of medical experiments (Turner et al., 



2012). It has been endorsed by over 600 journals (Moher, 
Altman, Schulz, & Elbourne, 2004), and it is supported by the 
Institute of Educational Sciences (Torgerson et al., 2005). 
CONSORT is the only guideline for reporting RCTs that has 
been developed with such rigor, and it has remained more pro- 
minent that any other guideline for over 15 years; for greatest 
impact, any further reporting guidelines related to RCTs should 
be developed in collaboration with the CONSORT Group. 

Limitations of Previous Reporting Guidelines for Social 
and Psychological Interventions 

Researchers and journal editors in the social and behavioral 
sciences are generally aware of CONSORT but often object 
that it is not fully appropriate for social and psychological inter- 
ventions (Bonell et al, 2006; Davidson et al, 2003; Perry et al., 
2010; Stinson et al., 2003). As a result, uptake of CONSORT 
guidelines in these disciplines is low. While some criticisms are 
due to inaccurate perceptions about common features of RCTs 
across disciplines, many relate to real limitations for social and 
psychological interventions (Mayo-Wilson, 2007). For exam- 
ple, CONSORT is most relevant to RCTs in medical disci- 
plines; it was developed by biostatisticians and medical 
researchers with minimal input from experts in other disci- 
plines. Journal editors, as well as social and behavioral science 
researchers, believe there is a need to include appropriate sta- 
keholders in developing a new, targeted guideline to improve 
uptake in their disciplines (Gill, 2011; Torgerson et al., 
2005). The CONSORT Group has produced extensions of the 
original CONSORT Statement relevant to social and psycholo- 
gical interventions, such as additional checklists for cluster 
(Campbell, Elbourne, & Altman, 2004), nonpharmacological 
(Boutron et al., 2008a), pragmatic (Zwarenstein et al., 2008), 
and quality of life RCTs (Calvert, Blazeby, Revicki, Moher, 
& Brundage, 2011). These extensions provide important 
insights, but complex social and psychological interventions, 
for example, include multiple, interacting components at sev- 
eral levels, with various outcomes. These RCTs require use 
of several extensions at once, creating a barrier to guideline 
uptake; increasing intervention complexity also gives rise to 
new issues that are not included in existing guidelines. There- 
fore, simply disseminating CONSORT guidelines as they stand 
is insufficient, as this would not address the need for editors 
and authors to "buy-in" to this process. To improve uptake 
in these disciplines, CONSORT guidelines need to be extended 
to specifically address the important features of social and 
psychological interventions. 

Social and behavioral scientists have developed other 
reporting guidelines, including the Workgroup for Intervention 
Development and Evaluation Research (WIDER) Recommen- 
dations for behavioral change interventions (Abraham, for the 
WIDER, 2009; Michie et al., 201 1), the American Educational 
Research Association's (AERA, 2006) Standards for Reporting 
Research, the REPOrting of Studies in Education (REPOSE) 
guidelines for primary research in education (Newman & 
Elbourne, 2004), and the Journal Article Reporting Standards 
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(JARS) of the American Psychological Association (APA) 
Publications and Communications Board Working Group on 
JARS (2008). While they address issues not covered by the 
CONSORT Statement and its extensions, these guidelines 
(except for JARS; APA Publications and Communications 
Board Working Group on JARS, 2008) do not provide specific 
guidance for RCTs. Moreover, compared with the CONSORT 
Statement and its official extensions, guidelines in the social 
and behavioral sciences have not consistently followed optimal 
techniques for guideline development and dissemination that 
are recommended by international leaders in the advancement 
of reporting guidelines (Moher, Schulz, et al., 2010), such as 
the use of systematic literature reviews and formal consensus 
methods to select reporting standards (Grant, Montgomery, & 
Mayo- Wilson, 2012). Researchers in public health, psychol- 
ogy, education, social work, and criminology have noted that 
these guidelines could be more "user-friendly," and dissemi- 
nation could benefit from up-to-date knowledge transfer tech- 
niques (Abraham, 2009; Armstrong et al, 2008; Davidson 
et al, 2003; Naleppa & Cagle, 2010; Perry & Johnson, 2008; 
Stinson et al, 2003; Torgerson et al., 2005). 

For example, JARS — a notable and valuable guideline for 
empirical psychological research — is endorsed by few journals 
outside of the APA, whereas CONSORT is endorsed by 
hundreds of journals internationally. According to ISI Web of 
Knowledge and Google Scholar citations, JARS is cited approx- 
imately a dozen times annually, while CONSORT guidelines are 
cited hundreds of times per year. Moreover, the APA commis- 
sioned a select group of APA journal editors and reviewers to 
develop JARS, and the group based most of their work on exis- 
tent CONSORT guidelines; by comparison, official CONSORT 
extensions have been developed using rigorous consensus meth- 
ods, have involved various international stakeholders in guide- 
line development and dissemination, and update content on the 
most recent scientific literature. Nonetheless, no current CON- 
SORT guideline adequately addresses the unique features of 
social and psychological interventions. This new CONSORT 
extension will incorporate lessons from previous extensions, 
reporting guidelines, and the research literature to aid the critical 
appraisal, replication, and uptake of this research. 

Aspects of Internal Validity 

Internal validity is the extent to which the results of a study 
may be influenced by bias. Like other study designs, the validity 
of RCTs depends on high-quality execution. Poorly conducted 
RCTs can produce more biased results than well-conducted 
RCTs and well-conducted nonrandomized studies (Pildal 
et al., 2007; Prescott et al., 1999). For example, evidence indi- 
cates that RCTs that do not adequately conceal the randomiza- 
tion sequence can exaggerate effect estimates by up to 30% 
(Schulz, Chalmers, Hayes, & Altman, 1995), while low- 
quality reports of these RCTs are associated with effect esti- 
mates exaggerated by up to 35% (Moher et al., 1999). Social 
and psychological intervention RCTs are susceptible to these 
risks of bias as well. 



Some aspects of internal validity, although included in 
CONSORT, remain poorly reported — even in the least com- 
plex social and psychological intervention studies. Reports 
of RCTs should describe procedures for minimizing selection 
bias, but reports often omit information about random 
sequence generation and allocation concealment (Ladd, 
McCrady, Manuel, & Campbell, 2010; Perry & Johnson, 
2008), and psychological journals report methods of sequence 
generation less frequently than medical journals (Stinson 
et al., 2003). A review of educational reports found no studies 
that adequately reported allocation concealment (Torgerson 
et al., 2005), and reports in criminology often lack informa- 
tion about randomization procedures (Gill, 2011; Perry 
et al., 2010). RCTs of social and psychological interventions 
may also use nontraditional randomization techniques, such 
as stepped wedge or natural allocation (MRC, 2011), which 
need to be thoroughly described. In addition, reports of social 
and psychological intervention trials often fail to include 
details about trial registration, protocols, and adverse events 
(Ladd et al., 2010; Perry & Johnson, 2008), which may 
include important negative consequences at individual, famil- 
ial, and community levels. 

Other aspects of CONSORT may require greater emphasis 
or modification for RCTs of social and psychological interven- 
tions. In developing this CONSORT extension, we expect to 
identify new items and to adapt existing items that relate to the 
internal validity. These may include items discussed during the 
development of previous CONSORT extensions or other 
guidelines, as well as items suggested by participants in this 
project. For example, it may not be possible to blind partici- 
pants and providers of interventions, but blinding of outcome 
assessors is often possible but rarely reported, and few studies 
explain if blinding was maintained or how lack of blinding was 
handled (Davidson et al., 2003; Ladd et al, 2010; Perry & 
Johnson, 2008). In social and psychological intervention stud- 
ies, outcome measures are often subjective, variables may 
relate to latent constructs, and information may come from 
multiple sources (e.g., participants and providers). While an 
issue in other areas of research, the influence on RCT results 
of the quality of subjective outcome measures in social and 
psychological intervention research has long been highlighted, 
given their prevalence in social and psychological intervention 
research (Marshall et al., 2000). Descriptions of the validity, 
reliability, and psychometric properties of such measures are 
therefore particularly useful for social and psychological inter- 
vention trials, especially when they are not widely available or 
discussed in the research literature (Campbell et al., 2004; 
Fraser, Galinsky, Richman, & Day, 2009). Moreover, multiple 
measures may be analyzed in several ways, so authors need to 
transparently report which procedures were performed and to 
explain their rationale. 

Aspects of External Validity 

External validity is the extent to which a study's results are 
applicable in other settings or populations. Currently, given 
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that RCTs are primarily designed to increase the internal valid- 
ity of study findings, the CONSORT Statement gives relatively 
little attention to external validity. While high internal validity 
is an important precondition for any discussion of an RCT's 
external validity, updating the CONSORT Statement to include 
more information about external validity is critical for the rele- 
vance and uptake of a CONSORT extension for social and 
psychological interventions. These interventions may be influ- 
enced by context, as different underlying social, institutional, 
psychological, and physical structures may yield different 
causal and probabilistic relations between interventions and 
observed outcomes. Contextual information is necessary to 
compare the effectiveness of an intervention across time and 
place (Cartwright & Munro, 2010). Lack of information rele- 
vant to external validity may prevent practitioners or policy 
makers from using evidence appropriately to inform decision 
making; yet, existing guidelines do not adequately explain how 
authors should describe (a) how interventions work, (b) for 
whom, and (c) under what conditions (Moore & Moore, 201 1). 

First, it is useful for authors to explain the key components 
of interventions, how those components could be delivered, 
and how they relate to the outcomes selected. At present, 
authors can follow current standards for reporting interventions 
without providing adequate details about complex interven- 
tions (Shepperd et al., 2009). Many reports neither contain 
sufficient information about the interventions tested nor refer- 
ence treatment manuals (Glasziou, Meats, Heneghan, & Shep- 
perd, 2008). Providing logic models — as described in the MRC 
Framework for Complex Interventions (Craig et al, 2008) — or 
presenting theories of change can help elucidate links in causal 
chains that can be tested, identify important mediators and 
moderators, and facilitate syntheses in reviews (Ivers et al., 
2012). Moreover, interventions are rarely implemented exactly 
as designed, and complex interventions may be designed to be 
implemented with some flexibility, in order to accommodate 
differences across participants (Hawe et al., 2004), so it is 
important to report how interventions were actually delivered 
by providers and actually received by participants (Hardeman 
et al., 2008). Particularly for social and psychological interven- 
tions, the integrity of implementing the intended functions and 
processes of the intervention are essential to understand (Hawe 
et al., 2004). As RCTs of a particular intervention can yield dif- 
ferent relative effects depending on the nature of the control 
groups, information about delivery and uptake should be pro- 
vided for all trial arms (McGrath, Stinson, & Davidson, 2003). 

Second, reports should describe recruitment processes and 
representativeness of samples. Participants in RCTs of social 
and psychological intervention are often recruited outside of 
routine practice settings via processes that differ from routine 
services (AERA, 2006). An intervention that works for one 
group of people may not work for people living in different 
cultures or physical spaces, or it may not work for people with 
slightly different problems and comorbidities. Enrolling in an 
RCT can be a complex process that affects the measured and 
unmeasured characteristics of participants, and recruitment 
may differ from how users normally access interventions. 



Well-described RCT reports will include the characteristics 
of all participants (volunteers, those who enrolled, and those 
who completed) in sufficient detail for readers to assess the 
comparability of the study sample to populations and in every- 
day services (AERA, 2006; APA Publications and Communi- 
cations Board Working Group on JARS, 2008; Evans & 
Brown, 2003) 

Finally, given that these interventions often occur in social 
environments, reports should describe factors of the RCT con- 
text that are believed to support, attenuate, or frustrate observed 
effects (Moore, 2002). Interventions may differ across groups 
of different social or socioeconomic positions, and equity 
considerations should be addressed explicitly (Tugwell et al., 
2010; Welch et al., 2012). Several aspects of setting and imple- 
mentation may be important to consider, such as administrative 
support, staff training and supervision, organizational 
resources, the wider service system, and concurrent political 
or social events (Bonell et al., 2012; Fixsen, Naoom, Blase, 
Friedman, & Wallace, 2005; Shepperd et al, 2009; Wang, 
Moss, & Hiller, 2006). Reporting process evaluations may help 
understand mechanisms and outcomes. 

Developing a New CONSORT Extension 

This new reporting guideline for RCTs of social and psycholo- 
gical interventions will be an official extension of the 
CONSORT Statement. Optimally, it will help improve the 
reporting of these studies. Like other official CONSORT exten- 
sions (Boutron et al., 2008a; Campbell et al, 2004; Hopewell 
et al., 2008; Zwarenstein et al., 2008), this guideline will be 
integrated with the CONSORT Statement and previous exten- 
sions, and updates of the CONSORT Statement may incorpo- 
rate references to this extension. 

The project is being led by an international collaboration of 
researchers, methodologists, guideline developers, funders, 
service providers, journal editors, and consumer advocacy 
groups. We will be recruiting participants in a manner similar 
to other reporting guideline initiatives — identifying stake- 
holders through literature reviews, the project's International 
Advisory Group, and stakeholder-initiated interest in the proj- 
ect (Michie et al., 201 1; Schulz et al., 2010). We hope to recruit 
stakeholders with expertise from all related disciplines and 
regions of the world, including low- and middle-income coun- 
tries. Methodologists will identify items that relate to known 
sources of bias, and they will identify items that facilitate sys- 
tematic reviews and research synthesis. Funders will consider 
how the guideline can aid the assessment of grant applications 
for RCTs and methodological innovations in intervention eva- 
luation. Practitioners will identify information that can aid 
decision making. Journal editors will identify practical steps 
to implement the guideline and to ensure uptake. 

We will use consensus techniques to reduce bias in group 
decision making and to promote widespread guideline uptake 
and knowledge translation activities upon project completion 
(Murphy et al., 1998). Following rigorous reviews of existing 
guidelines and current reporting quality, we will conduct an 
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online Delphi process to identify a prioritized list of reporting 
items to consider for the extension. That is, we will invite a 
group of experts to electronically answer questions about 
reporting items and to suggest further questions. We will circu- 
late their feedback to the group and ask a second round of ques- 
tions. The Delphi process will capture a variety of international 
perspectives and allow participants to share their views anon- 
ymously. Following the Delphi process, we will host a consen- 
sus meeting to review the findings and to generate a list of 
minimal reporting standards, mirroring the development of pre- 
vious CONSORT guidelines (Boutron et al, 2008b; Schulz 
et al, 2010; Zwarenstein et al., 2008). 

Together, participants in this process will create a checklist 
of reporting items and a flowchart for reporting social and psy- 
chological intervention RCTs. In addition, we will develop an 
Explanation and Elaboration (E&E) document to explain the 
scientific rationale for each recommendation and to provide 
examples of clear reporting; a similar document was developed 
by the CONSORT group to help disseminate a better under- 
standing for each included checklist item (Moher, Hopewell, 
et al., 2010). This document will help persuade editors, authors, 
and funders of the importance of the guideline. It will be a 
useful pedagogical tool, helping students and researchers 
understand the methods for conducting RCTs of social and psy- 
chological interventions, and it will help authors meet the 
guideline requirements (Moher, Schulz, et al., 2010). 

The success of this project depends on widespread involve- 
ment and agreement among key international stakeholders in 
research, policy, and practice. For example, previous develo- 
pers have obtained guideline endorsement by journal editors 
who require authors and peer reviewers to use the guideline 
during article submission and who must enforce journal article 
word limits (Michie, Fixsen, Grimshaw, & Eccles, 2009). 
Many journal editors have already agreed to participate, and 
we hope other researchers and stakeholders will volunteer their 
time and expertise. 

Conclusion 

Reporting guidelines help us use scarce resources efficiently 
and ethically. RCTs are expensive, and the public have a right 
to expect returns on their investments through transparent, 
usable reports. When RCT reports cannot be used (for whatever 
reason), resources are wasted. Participants contribute their time 
and put themselves at risk of harm to generate evidence that 
will help others, and researchers should disseminate that infor- 
mation effectively (Davidson et al., 2003). Policy makers ben- 
efit from research when developing effective, affordable 
standards of practice and choosing which programs and ser- 
vices to fund. Administrators and managers are required to 
make contextually appropriate decisions. Transparent reporting 
of primary studies is essential for their inclusion in systematic 
reviews that inform these activities. For example, there is the 
need to determine if primary studies are comparable, examine 
biases within included studies, assess the generalizability of 
results, and implement effective interventions. Finally, we 



hope this guideline will reduce the effort and time required for 
authors to write reports of RCTs. 

RCTs are not the only valid method for evaluating interven- 
tions (Bonell et al., 201 1) nor are they the only type of research 
that would benefit from better reporting (Goldbeck & Vitiello, 
201 1). Colleagues have identified the importance of reporting 
standards for other types of research, including observational 
(von Elm et al., 2007), quasi-experimental (Des Jarlais, Lyles, 
Crepaz, & the TREND Group, 2004), and qualitative studies 
(Tong, Sainsbury, & Craig, 2007). This guideline is the first 
step toward improving reports of many designs for evaluating 
social and psychological interventions, which we hope will 
be addressed by this and future projects. We invite stakeholders 
from disciplines that frequently research these interventions to 
join this important effort and participate in guideline develop- 
ment by visiting our website, where they can find more infor- 
mation about the project, updates on its progress, and sign up 
to be involved (http://tinyurl.com/CONSORT-study). 
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